Enhanced visualization techniques using reconstructed time waveforms

ABSTRACT

A source video of a scene showing moving objects which may be located in a region of interest is filtered and processed by constructing a representation such as a frequency spectrum plot of some of the frequencies of motion in the scene or region of interest and enabling a selection of frequency peaks from which to generate reconstructed waveforms at a pixel level according to a time domain fabrication method or alternatively a time domain differentiation fabrication method, then applying the reconstructed waveform at each pixel to a reference frame to produce a modified video recording.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of priority to U.S. ProvisionalApplication No. 63/084,073, which was filed on Sep. 28, 2020, thecontents of which are fully incorporated herein by reference.

BACKGROUND

The measurement of dynamic motion from civil structures, machines, andliving beings using video recordings from video acquisition devices hasgained wide acceptance since 2010. The video acquisition devices, suchas but not limited to one or more video cameras, webcams, or digitalcameras integral in cell phones providing a source video, offer theadvantages of being a non-contact sensor and provides information frommillions of pixels simultaneously. In the visible light range, changesof the light intensity can be related to the motion of objects in thefield of view. In this case, the fundamental unit of measurement isdisplacement, and the available accuracy is a tenth of a mil or less.The application of mathematical techniques to amplify motion to visuallyexaggerate the subtle motion of one or more objects in a frame and theability to modify the frame rate on replay of the recorded video allowstechnicians to visually present the motion of concern and providespowerful arguments about what is happening and the need for repair todecision-makers who may have limited technical understanding of theunderlying physics of the fault condition.

The use of video recordings to capture vibration characteristics ofobjects in the scene of view is extremely powerful because in additionto the quantitative measurement of the time waveform or frequencyspectrum of the motion in the scene the analyst can visualize the motionof all the objects or structures in the scene. This visualization of themotion can be mathematically modified by amplifying or filtering thevideo and replaying the frequency(s) of interest at a speed that isoptimized for observation by the human eye. This allows the analyst topresent the specific motion of concern in a visually enhanced formatthat is easily understood by less technically skilled individuals.

There are several shortcomings associated with traditional filteringtechniques when applied to video data. The duration of recording, thesampling rate limitation of conventional cameras, and data storagelimitations create constraints that impact the visual quality of themathematically modified video. Most conventional cameras providesampling rates of less than 1000 Hz. The duration of the recordingdetermines the lowest frequency that can be analyzed in the data and thefrequency difference or resolution of the frequency spectrum. Thetraditional filtering techniques when applied to finite sets of data mayproduce distortions at the beginning and end of the data or envelopingthroughout the data set which distorts the visual representation of thereconstructed motion. Although slightly greater than two samples percycle is enough to mathematically identify a specific frequency at whichan object or objects are moving using frequency analysis methods, whenthis data is filtered to create a waveform at that frequency a fewsamples per cycle does not provide a visual presentation of the motionthat is easy to follow due to the small numbers of samples per cycleavailable. This will almost always occur when studying higherfrequencies due to the limitations of the sampling rate of the camera.One approach for improving these visual effects is to capture data atextremely high rates for long durations and when processing to throw outdata that is distorted. However, when capturing millions of pixels athigh sampling rate for extended periods, the data storage requirementsand computation times increase dramatically.

Accordingly, there is a need to isolate frequencies of interest in avideo recording and present reconstructed videos which reduce noise andallow low or high frequencies to be visually observed with a reasonablenumber of samples per cycle and without distortion effects from theprocessing techniques. Frequency domain filtering techniques are knownto produce distortion in the reconstructed time waveform if thefrequency being filtered does not have an exact number of cycles in thesampled data set. Although distortion and other negative effects can beoffset using a time domain reconstruction method, this approach can bevery computationally intensive if the waveform is constructed from eachfrequency line present in the spectral data.

In view of such limitations, alternate filtering methods which overcomethe limitations described above is needed and described in thisapplication. Accordingly, time domain waveform reconstructiontechniques, described herein according to multiple embodiments andalternatives, not only remove the distortion inherent in frequencydomain filtering techniques, but such novel techniques offer theoptional advantage of creating waveforms with a higher number of samplesper period; and thus, reconstructing the video with a higher resolutionthan the originally collected video. Such novel techniques, as describedherein, reduce the time associated with waveform reconstruction andreduce random noise in the recording. Such methods promote efficiency,for example when a user desires to filter a video depicting motion ofobjects or people to only one or a few frequencies of interest foranalysis.

SUMMARY

In some embodiments described herein, improved time domainreconstruction methods are provided, from which a new waveformassociated with motion depicted in a video can be reconstructed withparticular focus on one or more frequencies or frequency ranges ofinterest and based upon a higher number of samples per cycle. Thisnoticeably improves the visual quality of the filtered motion in thevideo. Additionally, reconstructed time waveforms with only the specificfrequencies or frequency ranges of interest greatly reduces noise andimproves the quality of the video. This supports larger amplificationfactors and minimizes the grainy visual effect that results whenunwanted noise is amplified. Accordingly, waveform reconstructionmethods provided herein are able to be made more efficientcomputationally than frequency domain techniques when only a fewfrequencies are being displayed in the reconstructed waveform.

BRIEF DESCRIPTION OF FIGURES

The figures accompanying and forming part of this specification areincluded to depict certain aspects of the embodiments described herein.A clearer understanding of the embodiments, and of methods and systemsprovided herein, will become more readily apparent by referring to theexemplary, and therefore non-limiting, aspects illustrated and shown inthe drawing figures. The original test waveforms shown in FIGS. 1 and 2are for comparison only and were constructed using LabView routines toadd two sine waves of different frequencies and calculate the frequencyspectrum. The test waveforms for all other figures were measured from alaboratory test rotor assembly.

FIG. 1A is the frequency spectrum of a signal composed from two sinewaves at 50 Hz and 225 Hz and FIG. 1B is a time waveform created usingfrequency-based filtering methods to isolate the frequency of interestat 50 Hz using LabView, which illustrates the typical distortion at thebeginning and end of the data.

FIG. 2A is the frequency spectrum of a signal composed from two sinewaves at 205 Hz and 225 Hz and FIG. 2B is a time waveform created usingfrequency-based filtering methods to isolate a single frequency ofinterest at 205 Hz using LabView, which illustrates the pervasivedistortion throughout the data.

FIG. 3A depicts original visual data of the rotor test assembly ascaptured by the camera, with an amplification factor of 75× applied, andthe grainy nature is evident in the single frame of the video when thenovel embodiments described herein are not used.

FIG. 3B is the waveform of the X-axis motion at the region of interestidentified by label 31 in FIG. 3A.

FIG. 3C shows the frequency spectrum of the X-axis motion at the regionof interest identified by label 31 in FIG. 3A with dominant peaks at 1×and 2× rotational speed, 19.33 Hz and 38.68 Hz, respectively.

FIG. 4A illustrates a graphical user interface that has been setup toperform a bandpass filter operation and remove all frequencies except38.68 Hz using frequency domain filtering techniques as illustrated onthe spectrum by graying out the excluded frequency ranges.

FIG. 4B is one frame of the filtered video identifying the ROI, labeled41, where the motion is being measured and shows that much of the noisehas been removed as reflected in the reduced grainy appearance ascompared to FIG. 3A.

FIGS. 4C and 4D show the waveform and frequency spectrum, respectively,of the X-axis motion at the ROI identified by label 41 after removingall frequencies except 38.68 Hz using frequency domain filteringtechniques.

FIG. 5A illustrates a graphical user interface that has been setup toremove all frequencies except 38.68 Hz using the HDR filtering mode withthe same resolution as the original recording as indicated by the Factorequal to 1.0 as shown by label 51, as illustrated on the frequencyspectrum in this figure by graying out the excluded frequency ranges.

FIG. 5B is one frame of the filtered video identifying the ROI, labeledas 52, where the motion is measured and shows that much of the noise hasbeen removed as reflected in the reduced grainy appearance as comparedto FIG. 3A.

FIG. 5C is the waveform of the X-axis motion at the region of interestidentified by label 52 in FIG. 5B.

FIG. 6A illustrates a graphical user interface that has been setup toremove all frequencies except 38.68 Hz using the HDR filtering mode withenhanced resolution over the original recording as indicated by theFactor equal to 5.0 as shown by label 61, as illustrated on thefrequency spectrum in this figure by graying out the excluded frequencyranges.

FIG. 6B is one frame of the filtered video identifying the ROI, labeledas 62, where the motion is measured and shows that much of the noise hasbeen removed as reflected in the reduced grainy appearance as comparedto FIG. 3A.

FIG. 6C is the waveform of the X-axis motion at the region of interestidentified by label 62 in FIG. 6B and using 5 times the originalresolution to reconstruct the video and waveform.

MULTIPLE EMBODIMENTS AND ALTERNATIVES

The reconstruction of a time waveform using inverse frequency domaintransforms such as FFT convolution or simply windowing and applying theinverse FFT produces artifacts in the resulting time waveform data. Thisis illustrated using LabView program where a waveform is constructedfrom adding two sine waves of different frequencies and calculating thefrequency spectrum shown in FIG. 1A. As expected, there are two peaks inthis spectrum located at the respective frequencies, 50 Hz and 225 Hz,of the original sine wave components. If this data is filtered byremoving the frequency peaks at 225 Hz, and the waveform isreconstructed using an inverse FFT or FFT convolution, then the desiredresult is a single sine wave at 50 Hz. However, the process ofreconstruction produces distortion artifacts at the beginning and end ofthe waveform as illustrated in FIG. 1B. A second case shown in FIGS. 2Aand 2B uses a synthesized waveform formed by adding sine waves at 205 Hzand 225 Hz. When the frequency peak at 225 Hz is removed from thespectrum, the waveform constructed using the inverse FFT or FFTconvolution produces a distorted waveform with both artifacts at thebeginning and end of the waveform but also modulation of the sine waveamplitude throughout the entire data set. The desired result from thereconstructed waveform in both cases is a sine wave of constantamplitude at the single remaining frequency after the removal of the 225Hz peak.

A second approach using time domain techniques to reconstruct thewaveform would be to create a summation of sine waves from theamplitude, frequency, and phase of every frequency point in thespectrum. This approach is very inefficient computationally, thuslimiting its use. A variation of this approach is computationallypractical if only the larger peaks present in the spectrum are utilizedto reconstruct the waveform. Often, the user is interested in isolatingonly one frequency at a time so that he can visualize the effect of thatsingle frequency. In some instances, a user could have a prioriknowledge of the frequencies which he wants to investigate. However, itis more common that the user creates a composite frequency spectrum ortable as described in the provisional patent application 63/043,299,“Enhanced Analysis Techniques Using Composite Frequency Spectrum Data,”filed Jun. 24, 2020, the contents of which are fully incorporated hereinby reference. Alternately, the user may select a region of interest,ROI, from a single frame of the video and the software will produce timewaveform and frequency spectrum plots of the X-axis and Y-axis motion inthe ROI. It is understood that multiple methods are well known in theart for interrogating a video to measure motion. Most often, the userwill select a single frequency to filter. At other times, the user mayselect more than one frequency such as a fundamental frequency and oneor more associated harmonic frequencies to visualize. The selection ofthe frequencies to be filtered and whether the frequencies are displayedin a single video or in sequential videos with only a single frequencypresent is under the control of the user.

In one aspect, the frequency, amplitude, and phase are determined foreach pixel based upon changes in light intensity at each pixel overtime. This can be accomplished for example by applying a FourierTransform on each pixel with the individual intensity value of the pixelfrom each video frame comprising the time domain signal. For a givenfrequency, an amplitude and phase can be determined. The time waveformof a pure sine wave can be constructed from this data without the needto perform an inverse FFT. The elimination of this step could reduceprocessing time both computationally and alleviate requirements to movedata or allocate memory for this step. This reconstructed waveform canreplace the time domain data in the original images. A DC value for eachFourier Transform may be used for the offset or values from anindividual frame. The reconstructed waveform at each pixel is applied toa reference frame to produce a modified video recording. As persons ofordinary skill in the art will appreciate, other transforms rather thanthe Fourier Transform functions could be used whose basis functions arenot sine waves. In some cases, these alternate transforms may beadvantageous depending on the dynamics of the motion present in thedata. Other functions could be used instead of pure sine waves. Forexample, square waves or sawtooth waves could be used. For example, insome cases a pixel may undergo nearly on/off modulation. If a black andwhite region of an image moves completely past a pixel the result wouldbe a transition of nearly black to white more approximating a squarewave with the duty cycle depend on the time it spends imaging the blackor white portion of the region being imaged. In other cases, thevibration may not be purely sinusoidal with the modulation in intensitydue to the motion of the imaged area changing from white to black fasterthan when it modulates from black to white or vice versa. An evaluationof the motion data such as analysis of the spectrum could be used toindicated which waveform is appropriate for each pixel. A non-limitingexample would be to correlate the measured time waveform at each pixelwith a series of templates such as a square wave, triangular wave,sinusoidal wave, or other pattern. The template that correlates thehighest with the measured waveform could be used. Other factors such asareas that undergo saturation in the image could be deemed to be bestused with a flat top wave to account for the saturation.

As a non-limiting description, a waveform fabrication method employedaccording to multiple embodiments is as follows:

1. The user identifies one or more frequency peaks to be filtered from afrequency spectrum of a table of the N largest peaks located in thefrequency spectrum from a region of interest or composite spectral data.

2. Locate a more precise value of the nominal amplitude and frequencyvalues determined by the FFT algorithm for the set of peaks identifiedby the user by using interpolation, fitting techniques, or locationalgorithms based on the spectral windowing function applied to thesampled data before the FFT is performed.

3. Locate more precise phase values for the identified peaks in the setby adjusting the phase calculated, PC, from the real and imaginaryvalues of the FFT algorithm for the frequency line closest to the exactfrequency, using the formulas, {1} and {2}, below:X=(FE−FL)/FR  {1} where X should range from −0.5 to +0.5 and

-   -   FE=Exact Frequency    -   FL=Frequency of nearest frequency line in spectrum    -   FR=Frequency Resolution of spectrum        The Accurate Phase (in radians), PA=(PC−180*X+90)/57.2958  {2}

4. The new waveform, TWF, is calculated for the user-identified peaks,using the formula, {3}, below:TWF(n)=ΣA(i)*sin(2πF*T(n)+PA)  {3} where

-   -   A=Accurate amplitude of peak i    -   F=Accurate frequency of peak i    -   PA=Accurate phase of peak i, and    -   T(n) varies from 0 to the original total data collection time or        a user selected time period

5. The number of time steps, n=1 to S, in the fabricated waveform can beequal to the number of data samples/frames originally collected,T(n)=n*Delta-Time where n varies from 0, 1, 2 . . . SS=Duration*Sampling-Rate−1 and Delta-Time=1/Sampling Rate

Alternately, it can be increased such that there are more samples percycles in the new waveform by selecting a number that is a multiple ofS.Delta-Time=Duration/(k*S)T(n)=n*Delta-Time where n varies from 0, 1, 2 . . . (k*S)

This is particularly valuable when visualizing higher frequency motionto present smooth transitions of the repetitive motion rather than onlya few samples per cycle.

Beginning with FIG. 3A and progressing through FIG. 6C, data collectedfrom a rotor test assembly provides a good visual example of thedifference between the frequency domain and time domain reconstructiontechniques. FIG. 3A is comparative in nature and shows one frame of thevideo collected from rotor test assembly reprocessed with a factor of 75applied to amplify the motion of the assembly. There is no filteringapplied to this video and the grainy nature of the image is due to theamplification of the random noise in the video signal. The waveform andfrequency spectrum of the X-axis motion at the region of interest shownon the image as a red rectangle and labeled 31 is presented in FIG. 3B.The frequency spectrum has dominant peaks at 1× and 2× rotational speed,19.33 Hz and 38.68 Hz (and labeled in the figure as peaks 19 and 38),respectively.

FIG. 4A shows a setup screen used to apply a bandpass filter to thevideo at 38.68 Hz using frequency domain techniques. FIG. 4B is oneframe of the filtered video and shows that much of the noise has beenremoved as reflected in the reduced grainy appearance as compared toFIG. 3A. FIGS. 4C and 4D show the waveform and frequency spectrum,respectively, of the X-axis motion at the region of interest identifiedby label 41 after removing all frequencies except peak 38 (38.68 Hz)using frequency domain filtering techniques. Although the frequencyspectrum does demonstrate that the other frequencies in the spectrumhave been removed, the waveform shows a considerable amount of variationin the amplitude of the remaining sine wave at 38.68 Hz. The time domainwaveform reconstruction filtering method, denoted as the “HDR method” inthe software is the option selected in the field for “Filtering Mode” inFIGS. 5A and 6A, was applied to this same video of the rotor testassembly.

More specifically, FIG. 5A shows a setup for the filter operation toremove all frequencies except 38.68 Hz using the HDR filtering mode withthe same resolution as the original recording as indicated by the Factorequal to 1.0 as shown by label 51. FIG. 5B is one frame of the filteredvideo and shows that much of the noise has been removed as reflected inthe reduced grainy appearance as compared to FIG. 3A. FIG. 5C shows thewaveform of the X-axis motion at the region of interest identified bylabel 52 in FIG. 5B after removing all frequencies except 38.68 Hz usingtime domain HDR mode filtering techniques. The waveform reconstructedusing the time domain fabrication method with the same resolution as theoriginal data and shows no undesirable signal processing distortion, butthe time waveform still exhibits some variation in the amplitude of the38.68 Hz sine wave due to the limited number of samples available foreach cycle. This is an improvement over the waveform in 4C. To reproducethe sine wave with very little apparent variation in amplitude thewaveform needs to be reconstructed with a higher number of samples percycle of the 38.68 Hz sine wave.

Continuing further, FIG. 6A is the setup for the filter operation toremove all frequencies except 38.68 Hz using the HDR filtering mode with5 times the original resolution of the original recording as indicatedby the Factor equal to 5.0 as shown by label 61. FIG. 6B is one frame ofthe filtered video and shows that much of the noise has been removed asreflected in the reduced grainy appearance as compared to FIG. 3A. FIG.6C shows the waveform of the X-axis motion at the region of interestidentified by label 62 in FIG. 6B after removing all frequencies except38.68 Hz using time domain HDR mode filtering techniques and using 5times the original resolution to construct the video and waveform.

Accordingly, aspects of the present embodiments as described hereininclude removal of noise and increasing the samples per cycle to enhanceresolution. The reconstructed waveform from a single frequency is a puresine wave. When this approach is applied to create a waveform of theintensity changes at each pixel in the scene of the video and thechanges in the waveform are applied to a reference frame, then themodified video of the frequency-specific motion in the scene is notdistorted by the fabrication process and significantly reduces the noisein the video. This allows larger values of amplification to be appliedwithout degrading the video due to noise saturation. The reconstructionprocess removes the noise from all pixels at all frequencies where apeak was not present in the frequency spectrum of that pixel. Thissubstantially improves the clarity of the resulting video.

FIGS. 5C and 6C compare two waveforms reconstructed using the timedomain fabrication method for a higher frequency peak at 38.68 Hzsampled at 121 samples per second shown first with the same number ofsamples as the original data set in the source video and then with 5times the number of samples as the original data illustrating theimproved ability to visualize the motion. The original data sampled at121 Hz provides less than 4 sample per cycle of the 38.68 Hz frequency.This produces a very sparse, choppy view of this motion at thisfrequency. The waveform constructed with the resolution increased by afactor of 5 results in about 15 samples per cycle and provides anexcellent visual presentation of this motion. FIGS. 5B and 6B presents asingle frame of the video reconstructed using a time domain fabricationmethod and an amplification factor of 75 showing the improved visualquality of the video. The same video has much greater clarity as aresult of the noise reduction inherent in this method of reconstruction.Of course, the impact is best seen when viewing the video, but a singleframe still illustrates the improved clarity. The removal of noiseallows higher amplification factors to be applied to the video with muchlower levels of noise contamination.

This technique can be applied by a user who is selecting one or morefrequencies or frequency ranges for study individually or incombination. Accordingly, in some embodiments, a system to evaluatemoving objects employs a computer program in a processor to: construct arepresentation of the plurality of frequencies at a selected location inthe source video whose displacements exceeds a selected threshold;provide a graphical user interface to enable a user to select one ormore of the plurality of frequencies to be included in a plurality ofreconstructed waveforms; generate a reconstructed waveform at each pixelusing a time domain fabrication method constructed from a summation ofsine waves from the amplitude, frequency, and phase of the selectedfrequencies, wherein the number of samples is equal to or greater thanthe number of samples for the source video; and apply the reconstructedwaveform at each pixel to a reference frame to produce a modified videorecording. In some embodiments, the computer program further operates insaid processor to amplify motion of at least one of the moving objectsas displayed in the modified video recording. An example of processingmethods to generate modified video images to amplify motion and enhancethe visual depictions from a source video is found in US Pub. No.2016/0300341 titled “Apparatus and Method for Visualizing PeriodicMotions in Mechanical Components” (Hay, Jeffrey R. et al.; publishedOct. 13, 2016), now patented as U.S. Pat. No. 10,062,411, the contentsof which are fully incorporated by reference herein.

Additionally, the software could automatically step through a subset ofthe largest peaks and create a filtered video for each frequency. Thesubset of frequencies could be a number, K, of largest peaks in thefrequency spectrum which are either selected or determined based onexceeding a threshold, or a user-selected subset from a table of thelargest peaks present in the spectrum. Similarly, the software couldidentify families of harmonic or sideband peaks and process thisplurality of peaks to create a video for each harmonic or sidebandfamily. The methods for locating harmonic or sideband families of peaksin a spectrum is well known to those skilled in the art.

A variation of this method can be used to improve the quality of adifferentiated video which presents the motion in the video in terms ofvelocity or acceleration rather the native displacement units. Asappreciated by persons of ordinary skill in the art, a common method ofdifferentiating motion data is to multiply the value of each line in thespectrum by a differentiation factor equal to 2π times the frequencyvalue (2πF) to get velocity and by the square of 2πF to obtainacceleration. Thus, the velocity or acceleration video of the selectedfrequencies could be obtained by multiplying the amplitude of the sinewave by the appropriate differentiation factor using the more preciseamplitude and frequency values of the selected peaks.

In some cases, the user might want to see the overall motion of thevideo in velocity or acceleration rather than just a few selectedfrequency peaks. Accordingly, in some embodiments, a time domaindifferentiation fabrication method is performed as follows:

1. Locate the largest N peaks in the frequency spectrum above the noisefloor of the spectrum for one or more spatial locations in the video.

2. Determine the maximum peak amplitude in the set of located peaks andpreferably, discard any peaks in the set that are a factor of K lessthan the maximum peak amplitude since the motion associated with verysmall peaks may result from noise. This results in a set of M remainingpeaks.

3. Locate a more precise value of the amplitude and frequency valueprovided by the FFT algorithm for the set of M remaining peaks usinginterpolation, fitting techniques, or location algorithms based on thespectral windowing function applied to the sampled data before the FFTis performed.

4. Locate more precise phase values, PAO, for the remaining peaks in theset by adjusting the phase calculated by the FFT algorithm, PC, from thereal and imaginary values of the spectrum for the frequency line closestto the exact frequency, using the formulas, {4} and {5}, below:X=(FE−FL)/FR  {4} where X should range from −0.5 to +0.5 and

-   -   FE=Exact Frequency    -   FL=Frequency of nearest frequency line in spectrum    -   FR=Frequency Resolution of spectrum        The Accurate Phase in radians, PAO=(PC−180*X+90)/57.2958  {5}

5. Shift the phase by a phase differentiation factor, PIF, which is 90degrees (1.57 radians) for conversion to velocity and 180 degrees (3.14radians) for conversion to acceleration,PA=PAO+PIF

6. The new waveform, TWF, is calculated by formula {6} using anamplitude differentiation factor, D, for the M remaining peaks in theset of located peaksTWF(n)=ΣD*A(i)*sin(2πF*T(n)+PA)  {6} where

-   -   D=2πF for velocity or (2πF) squared for acceleration    -   A=Accurate amplitude of peak i    -   F=Accurate frequency of peak i    -   PA=Accurate phase of peak i, and    -   T(n) varies from 0 to the original total data collection time or        a user selected time period

The number of time steps, n=1 to S, in the fabricated waveform can beequal to the number of data samples/frames originally collected,T(n)=n*Delta-Time where n varies from 0, 1, 2 . . . SS=Duration*Sampling-Rate−1 and Delta-Time=1/Sampling Rate

Alternately, it can be increased such that there are more samples percycles in the new waveform by selecting a number that is a multiple ofS.Delta-Time=Duration/(k*S)T(n)=n*Delta-Time where n varies from 0, 1, 2 . . . (k*S)

This is particularly valuable when visualizing higher frequency motionto present smooth transitions of the repetitive motion rather than on afew samples per cycle.

Although the exemplary method described above addresses differentiatinga displacement waveform to obtain a velocity or acceleration waveform,this technique could also be applied in the reverse sense to integratean acceleration waveform to obtain a velocity or displacement waveform.In this case, the waveform is multiplied by the amplitude integrationfactor, 1/D:TWF(n)=1/D*A(i)*sin(2πF*T(n)+PA)

Also, the integration phase shift, PIF, is subtracted from thecalculated phase rather than added:PA=PAO−PIF

Accordingly, in some embodiments, a system to evaluate moving objectsemploys a computer program in a processor to: calculate a frequencyspectrum at one or more spatial locations in the source video toidentify the plurality of frequencies present in the video; locate the Nlargest peaks among the plurality of frequencies above the noise floorof the one or more frequency spectra; discard one or more frequencypeaks among the plurality of frequencies that are at or below a factorof K less than the maximum peak, resulting in a set of M remainingfrequency peaks to be included in a reconstructed waveform for eachpixel; generate a reconstructed waveform at each pixel using a timedomain differentiation fabrication method constructed from a summationof sine waves using a modified amplitude and phase of each remainingfrequency wherein the amplitude and phase are modified by adifferentiation factor and wherein the number of samples is equal to orgreater than the number of samples for the source video; and apply thereconstructed waveform at each pixel to a reference frame to produce amodified video recording.

Multiple embodiments and alternatives disclosed herein may be recited orpracticed as methods, or as part of a system for evaluating a collectionof moving objects undergoing periodic motion and depicted in a videorecording. In some embodiments, a new waveform is constructed which isspecific to a selected frequency or frequencies. A reconstructed videowith less noise can be created by applying such new waveform to areference frame obtained from the video recording. Various alternativeapproaches are provided for fabricating the new waveform, which involvemanipulation of amplitude, frequency, and phase characteristics of themotion in question. In some embodiments, results are improved, andresolution is enhanced by increasing the number of data samples percycle. In view of the embodiments described herein, filtering techniquesmay be limited to specific objects, object types or other limitingsubject matter. Image segmentation may be used to narrow a region ofinterest or pixels where this technique will be applied. It will beappreciated that image segmentation is a well-known and establishedtechnique, which is familiar to persons of ordinary skill in this field.The user may employ image segmentation by selecting an area in the imageand the segmentation process includes all areas associated with thatsegment. For example, the user may click on a pipe and the imagesegmentation process identifies all areas in the image associated withor a part of that pipe for inclusion. Object recognition may be anotherway in which the areas are define where this waveform fabricationtechnique is applied to a subset of pixels. It will be appreciated thatobject recognition methods are well-known and established techniquesknown to persons of ordinary skill in this field. For example, deeplearning models such as convolutional neural networks are used toautomatically learn an object's inherent features in order to identifythat object. Other machine learning methods are based on techniques suchas HOG feature extraction using an SVM learning model, a Bag-of-wordsmodel with features such as SURF and MSER, and the Viola-Jonesalgorithm. More basic object recognition methods are based on templatematching or image segmentation and blob analysis.

The user may wish to look at only the motor or pump or both. Objectrecognition may be employed to identify the pixels associated with themotor. The object recognition may identify the object, and the pixelsassociated with the object and only include that object. Objectrecognition may also be used to identify an object type, for example,pipes so that all pipes in the scene are included.

It will be understood that the embodiments described herein are notlimited in their application to the details of the teachings anddescriptions set forth, or as illustrated in the accompanying figures.Rather, it will be understood that the present embodiments andalternatives, as described and claimed herein, are capable of beingpracticed or carried out in various ways. Also, it is to be understoodthat words and phrases used herein are for the purpose of descriptionand should not be regarded as limiting. The use herein of such words andphrases as “including,” “such as,” “comprising,” “e.g.,” “containing,”or “having” and variations of those words is meant to encompass theitems listed thereafter, and equivalents of those, as well as additionalitems.

Accordingly, the foregoing descriptions of several embodiments andalternatives are meant to illustrate, rather than to serve as limits onthe scope of what has been disclosed herein. The descriptions herein arenot intended to be exhaustive, nor are they meant to limit theunderstanding of the embodiments to the precise forms disclosed. Interms of the descriptions, it will be understood by those havingordinary skill in the art that modifications and variations of theseembodiments are reasonably possible in light of the above teachings anddescriptions.

What is claimed is:
 1. A system for evaluating moving objects undergoingperiodic motion using at least one video acquisition device thatacquires sampling data as a plurality of video images of the movingobjects, with the video images being divisible into individual videoimage frames, and with each video image frame being divisible into aplurality of pixels, comprising: a video acquisition device configuredwith an adjustable frame rate to obtain a source video of the movingobjects, wherein the adjustable frame rate allows video images to beacquired at a sampling rate that is sufficient to capture a plurality offrequencies present in the periodic motion being evaluated; a processorand a memory for storage of the source video including individual videoimage frames; and a computer program operating in said processor to:construct a representation of the plurality of frequencies at one ormore locations in the source video whose displacements exceeds aselected threshold; provide a graphical user interface to enable a userto select one or more of the plurality of frequencies to be included ina plurality of reconstructed waveforms; generate a reconstructedwaveform at each pixel using a time domain fabrication methodconstructed from a summation of sine waves from an amplitude, frequency,and phase of the selected frequencies, wherein a number of samples isequal to or greater than the number of samples for the source video; andapply the reconstructed waveform at each pixel to a reference frame toproduce a modified video recording.
 2. The system of claim 1, whereinthe time domain fabrication method uses values of the amplitude,frequency, and phase for the plurality of frequencies present in thevideo which are more accurate than the nominal FFT values calculated forthese frequencies.
 3. The system of claim 1, wherein the computerprogram further operates in said processor to amplify motion of at leastone of the moving objects as displayed in the modified video recording.4. The system of claim 1, wherein the representation of the plurality offrequencies comprises a spectrum plot.
 5. The system of claim 1, whereinthe representation of the plurality of frequencies comprises a tableidentifying the plurality of frequencies.
 6. The system of claim 1,wherein the displacement motion in the modified video is converted tovelocity or acceleration presentation of the motion by applying theappropriate differentiation factor modifications to the amplitude andphase of the of the sine wave representation of the one or morefrequencies selected to be shown in the modified video.
 7. A system forevaluating moving objects undergoing periodic motion using at least onevideo acquisition device that acquires sampling data as a plurality ofvideo images of the moving objects, with the video images beingdivisible into individual video image frames, and with each video imageframe being divisible into a plurality of pixels, comprising: a videoacquisition device configured with an adjustable frame rate to obtain asource video of the moving objects, wherein the adjustable frame rateallows video images to be acquired at a sampling rate that is sufficientto capture a plurality of frequencies present in the periodic motionbeing evaluated; a processor and a memory for storage of the sourcevideo including individual video frames; and a computer programoperating in said processor to: calculate a frequency spectrum at one ormore spatial locations in the source video to identify the plurality offrequencies present in the video; locate the N largest peaks among theplurality of frequencies above a noise floor of the one or morefrequency spectra; discard one or more frequency peaks among theplurality of frequencies that are at or below a factor of K less than amaximum peak, resulting in a set of M remaining frequency peaks to beincluded in a reconstructed waveform for each pixel; generate areconstructed waveform at each pixel using a time domain differentiationfabrication method constructed from a summation of sine waves using amodified amplitude and phase of each remaining frequency wherein theamplitude and phase are modified by a differentiation factor and whereina number of samples is equal to or greater than the number of samplesfor the source video; and apply the reconstructed waveform at each pixelto a reference frame to produce a modified video recording.
 8. Thesystem of claim 7, wherein the time domain fabrication method usesvalues of the amplitude, frequency, and phase for the located peaks ofinterest which are more accurate than the nominal FFT values of theplurality of frequencies captured.
 9. The system of claim 7, wherein thecomputer program also operates in said processor to amplify motion of atleast one of the moving objects as displayed in the modified videorecording.
 10. The system of claim 7, wherein the computer programautomatically steps through a subset of the set of largest peaks andproduces individual filtered videos for each frequency in the subset.11. The system of claim 7, wherein the computer program integrates themotion in the source video presented in acceleration units to produce avideo whose motion is presented in velocity units, or the computerprogram integrates the motion in the source video presented in velocityunits to produce a video whose motion is presented in displacementunits.
 12. A system for evaluating moving objects undergoing periodicmotion using at least one video acquisition device that acquiressampling data as a plurality of video images of the moving objects, withthe video images being divisible into individual video image frames, andwith each video image frame being divisible into a plurality of pixels,comprising: a video acquisition device configured with an adjustableframe rate to obtain a source video of the moving objects, wherein theadjustable frame rate allows video images to be acquired at a samplingrate that is sufficient to capture a plurality of frequencies present inthe periodic motion being evaluated; a processor and a memory forstorage of the source video including individual video image frames; anda computer program operating in said processor to: construct arepresentation of the plurality of frequencies at one or more locationsin the source video whose displacements exceeds a selected threshold;automatically select one or more of the plurality of frequencies to beincluded in a plurality of reconstructed waveforms; generate areconstructed waveform at each pixel using a time domain fabricationmethod constructed from a summation of sine waves from an amplitude,frequency, and phase of the selected frequencies, wherein a number ofsamples is equal to or greater than the number of samples for the sourcevideo; and apply the reconstructed waveform at each pixel to a referenceframe to produce a modified video recording.
 13. The system of claim 12,wherein the computer program selects the frequencies to be processed byidentifying the largest N peaks, or by identifying families of harmonicor sideband peaks.
 14. The system of claim 12, wherein the time domainfabrication method uses values of the amplitude, frequency, and phasefor the plurality of frequencies present in the video which are moreaccurate than the nominal FFT values calculated for these frequencies.15. The system of claim 12, wherein the computer program furtheroperates in said processor to amplify motion of at least one of themoving objects as displayed in the modified video recording.
 16. Thesystem of claim 12, wherein the displacement motion in the modifiedvideo is converted to velocity or acceleration presentation of themotion by applying the appropriate differentiation factor modificationsto the amplitude and phase of the of the sine wave representation of theone or more frequencies selected to be shown in the modified video.